Omni-channel orchestrated conversation system and virtual conversation agent for realtime contextual and orchestrated omni-channel conversation with a human and an omni-channel orchestrated conversation process for conducting realtime contextual and fluid conversation with the human by the virtual conversation agent

ABSTRACT

An omni-channel orchestrated conversation system and virtual conversation agent for realtime contextual and orchestrated omni-channel conversation with a human and an omni-channel orchestrated conversation process for conducting realtime contextual and fluid conversation with a human by a virtual conversation agent in relation to a particular domain are disclosed.

CLAIM OF BENEFIT TO PRIOR APPLICATION

This application claims benefit to U.S. Provisional Patent Application63/176,043, entitled “FLUID CONVERSATION SYSTEM AND VIRTUAL SPEECH AGENTFOR REALTIME CONTEXTUAL AND FLUID SPEECH CONVERSATION WITH A HUMAN,”filed Apr. 16, 2021. The U.S. Provisional Patent Application 63/176,043is incorporated herein by reference.

BACKGROUND

Embodiments of the invention described in this specification relategenerally to virtual communication systems and virtual responsive textbots, and more particularly, to an omni-channel orchestratedconversation system and virtual conversation agent for realtimecontextual and orchestrated omni-channel conversation with a human andan omni-channel orchestrated conversation process for conductingrealtime contextual and fluid conversation with a human by a virtualconversation agent.

Existing virtual conversation systems, such as chat “bots” and the like,have various limitations that make them unsuitable for many types ofconversations. These limitations include, non-exhaustively, inability tohold a realtime, spontaneous and contextually relevant telephonic speechconversation with a human (e.g., not able to engage in a realtimetelephonic screening of a candidate for employment), inability toeffectively handle interruptions by the human speaker during theconversation, lack of domain-specific conversational data for languageprocessing, inability to engage in non-linear conversations, and theinability to ask contextually-relevant follow up questions whenresponses from the human speaker are insufficiently clear, detailed,and/or explained.

Therefore, what is needed is a way to provide fluid, orchestrated, andcontextual conversations with humans by a virtual conversation agentthat conducts fluid, human-like conversations by way of any of severalcommunication channels, including at least telephone, email, and textmessaging.

BRIEF DESCRIPTION

A novel omni-channel orchestrated conversation system and virtualconversation agent for realtime contextual and orchestrated omni-channelconversation with a human is disclosed. In some embodiments, the virtualconversation agent engages in contextual and fluid telephonic speechconversation with a human in relation to a particular domain. In someembodiments, the omni-channel orchestrated conversation system supportsconversational engagement between a virtual conversation agent and anactual human (such as a job candidate) over telephony, which differsfrom human-to-human conversational engagement. In particular, theomni-channel orchestrated conversation system provides intelligent,orchestrated conversation by an artificial intelligence engine supportedby a machine learning (ML) sub-system that trains the virtualconversation agent to conduct fluid, human-like conversations with ahuman across any of several channels (e.g., telephonic speech, textmessage, and email conversations). The virtual conversation agent isdesigned to simulate the way an actual human speaker would carry out atelephone-based conversation in a particular domain, or otherwise engagein a conversation (such as through email, text messaging in realtimechat, etc.), which helps the omni-channel orchestrated conversationsystem to advance efforts intended to achieve a goal in connection withthe human in the particular domain. In some embodiments, the virtualconversation agent is able to determine whether to ask contextuallyrelevant follow-up questions during the conversation with the humandepending on the quality of the human's prior response(s). In someembodiments, the virtual conversation agent generates a summary reportwhen the conversation with the human concludes with information derivedfrom the conversation that is relevant in the particular domain. In someembodiments, the information in the summary report is scored accordingto one or more aspects pertaining to the particular domain. Inparticular, the omni-channel orchestrated conversation system alsocalculates a candidate fit score.

A novel omni-channel orchestrated conversation process for conductingrealtime contextual and fluid conversation with a human by a virtualconversation agent is also disclosed. In some embodiments, theomni-channel orchestrated conversation process for conducting realtimecontextual and fluid conversation with a human by a virtual conversationagent involves a phone call conversation between the human and thevirtual conversation agent. In some embodiments, the omni-channelorchestrated conversation process for conducting realtime contextual andfluid conversation with a human by a virtual conversation agent involvesa text message-based conversation between the human and the virtualconversation agent. In some embodiments, the omni-channel orchestratedconversation process for conducting realtime contextual and fluidconversation with a human by a virtual conversation agent involves anemail-based conversation between the human and the virtual conversationagent. In some embodiments, the omni-channel orchestrated conversationprocess for conducting realtime contextual and fluid conversation with ahuman by a virtual conversation agent converges a text message-basedconversation between the human and the virtual conversation agent into aphone call conversation between the human and the virtual conversationagent. In some embodiments, the omni-channel orchestrated conversationprocess for conducting realtime contextual and fluid conversation with ahuman by a virtual conversation agent converges an email-basedconversation between the human and the virtual conversation agent into aphone call conversation between the human and the virtual conversationagent.

The preceding Summary is intended to serve as a brief introduction tosome embodiments of the invention. It is not meant to be an introductionor overview of all inventive subject matter disclosed in thisspecification. The Detailed Description that follows and the Drawingsthat are referred to in the Detailed Description will further describethe embodiments described in the Summary as well as other embodiments.Accordingly, to understand all the embodiments described by thisdocument, a full review of the Summary, Detailed Description, andDrawings is needed. Moreover, the claimed subject matters are not to belimited by the illustrative details in the Summary, DetailedDescription, and Drawings, but rather are to be defined by the appendedclaims, because the claimed subject matter can be embodied in otherspecific forms without departing from the spirit of the subject matter.

BRIEF DESCRIPTION OF THE DRAWINGS

Having described the invention in general terms, reference is now madeto the accompanying drawings, which are not necessarily drawn to scale,and wherein:

FIG. 1 conceptually illustrates an omni-channel orchestratedconversation process for conducting realtime contextual and orchestratedomni-channel conversation with a human by a virtual conversation agentin some embodiments.

FIG. 2 conceptually illustrates a continuation of the omni-channelorchestrated conversation process for conducting realtime contextual andorchestrated omni-channel conversation with a human by a virtualconversation agent of FIG. 1 .

FIG. 3 conceptually illustrates a continuation of the omni-channelorchestrated conversation process for conducting realtime contextual andorchestrated omni-channel conversation with a human by a virtualconversation agent of FIG. 2 .

FIG. 4 conceptually illustrates an omni-channel orchestratedconversation system with a virtual conversation agent for realtimecontextual and orchestrated omni-channel conversation with a human insome embodiments.

FIG. 5 conceptually illustrates an electronic system with which someembodiments of the invention are implemented.

DETAILED DESCRIPTION

In the following detailed description of the invention, numerousdetails, examples, and embodiments of the invention are described.However, it will be clear and apparent to one skilled in the art thatthe invention is not limited to the embodiments set forth and that theinvention can be adapted for any of several applications.

Some embodiments include an omni-channel orchestrated conversationsystem and virtual conversation agent for realtime contextual andorchestrated omni-channel conversation with a human. In someembodiments, the virtual conversation agent engages in contextual andfluid telephonic speech conversation with a human in relation to aparticular domain. In some embodiments, the omni-channel orchestratedconversation system supports virtual conversation agent to actual humanconversational engagement over telephony, which differs fromhuman-to-human conversational engagement. In this way, the omni-channelorchestrated conversation system provides the virtual conversation agentto conduct telephonic speech conversations in lieu of providing directtelephone contact with a live human agent. The virtual conversationagent is designed to simulate the way an actual human speaker wouldcarry out a telephone-based conversation in a particular domain, whichhelps the omni-channel orchestrated conversation system to advanceefforts intended to achieve a goal in connection with the human in theparticular domain. In some embodiments, the virtual conversation agentis able to determine whether to ask contextually relevant follow-upquestions during the telephonic speech conversation with the humandepending on the quality of the human's prior response(s). In someembodiments, the virtual conversation agent generates a summary reportwhen the telephonic speech conversation with the human concludes withinformation derived from the telephonic speech conversation that isrelevant in the particular domain. In some embodiments, the informationin the summary report is scored according to one or more aspectspertaining to the particular domain. An example of an omni-channelorchestrated conversation system and a virtual conversation agent isdescribed further below, by reference to FIG. 4 .

In some embodiments, the virtual domain-specific speech agent is avirtual speech recruiter and the particular domain is candidaterecruiting for a job (employment work, contractual work, and the like).In some embodiments, the virtual speech recruiter is used to conducttelephonic speech conversations in lieu of providing direct telephoniccontact with a live human recruiter. The omni-channel orchestratedconversation system is designed to simulate the way a human recruiterwould carry out a candidate screening telephone call, which then helpsto prioritize which candidate can be submitted for a job.

In some embodiments, the omni-channel orchestrated conversation systemsimulates the way a human would carry out a conversation with a human byway of an omni-channel orchestrated conversation process for conductingrealtime contextual and fluid conversation with a human by a virtualconversation agent. In some embodiments, the omni-channel orchestratedconversation process for conducting realtime contextual and fluidconversation with a human by a virtual conversation agent involves aphone call conversation between the human and the virtual conversationagent. In some embodiments, the omni-channel orchestrated conversationprocess for conducting realtime contextual and fluid conversation with ahuman by a virtual conversation agent involves a text message-basedconversation between the human and the virtual conversation agent. Insome embodiments, the omni-channel orchestrated conversation process forconducting realtime contextual and fluid conversation with a human by avirtual conversation agent involves an email-based conversation betweenthe human and the virtual conversation agent. In some embodiments, theomni-channel orchestrated conversation process for conducting realtimecontextual and fluid conversation with a human by a virtual conversationagent converges a text message-based conversation between the human andthe virtual conversation agent into a phone call conversation betweenthe human and the virtual conversation agent. In some embodiments, theomni-channel orchestrated conversation process for conducting realtimecontextual and fluid conversation with a human by a virtual conversationagent converges an email-based conversation between the human and thevirtual conversation agent into a phone call conversation between thehuman and the virtual conversation agent. An omni-channel orchestratedconversation process for conducting realtime contextual and fluidconversation with a human by a virtual conversation agent is describedin detail below, by reference to FIGS. 1-3 .

In some embodiments, the virtual conversation agent is used to conductconversations across different channels such as phone call, email, andtext in lieu of providing direct contact with a live human agent. Insome embodiments, the omni-channel orchestrated conversation system isdesigned to simulate the way a human agent (recruiter) would carry outfluid contextual conversations—including follow up conversations—acrossa variety of communication channels, such as voice (telephone), email,and text with another human (candidate screening). Systems in thecurrent arts have various limitations including the inability to hold areal time telephonic screening with a candidate, handle interruptions bythe candidate, limitations in language processing due to lack of domainspecific conversational data, inability to engage in non-linearconversations and the inability to ask contextual follow up questions.The existing systems also lack in the ability to automaticallyorchestrate follow up conversations across different channels based onthe context of current conversation on one channel. The virtual agentalso understands which person to have the conversation with by analyzingthe notes of past conversations between a human/virtual agent and theperson.

In some embodiments, the omni-channel orchestrated conversation systemsupports real time speech-based conversation over telephony where thevirtual conversation agent (or “virtual recruiter”) can understand thehuman's intent and navigate the conversation, thereby enabling the humancandidate at the other end to respond naturally. This naturalconversation opens the door for the human candidate to speak at lengthand describe their experiences in a way that they would only do whentalking to a real human. Follow-up probing questions that are askedbased on the response assessment of the previous questions, enables thecandidates to provide more specifics about their experiences. In someembodiments, the virtual conversation agent is also configured tonegotiate with the human candidate when the need arises. In the case ofcandidate screening, for example, the virtual conversation agent cannegotiate pay rates based on the configured pay range and the candidateexpectations during the conversation.

Furthermore, critical signals beyond candidate skills that are relevantfor hiring decisions are adequately captured by embodiments of theomni-channel orchestrated conversation system described in the presentspecification. Final output includes a well-constructed call screeningsummary that includes fitness scores, skills details, and business needssignals categorized into highlights, lowlights and considerationsempowers the recruiting team to rapidly make hiring decisions on thecandidate.

Thus, the omni-channel orchestrated conversation system and the virtualconversation agent described in this specification solve the problemsand limitations of existing virtual communication systems noted above.The intelligent, machine learning approach of the omni-channelorchestrated conversation system obviates the need to engage a livehuman agent to speak with another human since the virtual conversationagent is designed to simulate the way an actual human speaker wouldcarry out a telephonic speech conversation (or conversation via textmessaging or email) in a particular domain.

In some embodiments, the omni-channel orchestrated conversation systemand the virtual conversation agent make realtime speech basedconversation over telephony possible where the virtual conversationagent presents as a virtual recruiter who understand the human's intentand navigate the telephonic speech conversation in a seemingly naturalmanner which allows the human candidate at the other end of the call torespond naturally. This natural conversation opens the door for freeflow of information, whereby the candidate can speak at length anddescribe their experiences in a way that they would only do when talkingto a real human. Additionally, the virtual conversation agent isconfigured to utilize an artificial intelligence (AI) engine and amachine learning (ML) sub-system of the omni-channel orchestratedconversation system to seamlessly ask the candidate follow-up probingquestions that are based on the AI engine and ML sub-system assessmentsof the candidate's responses to the previous questions, thereby enablingthe candidate to elaborate further and provide more specifics abouttheir experiences. Furthermore, critical signals, beyond candidateskills, that are relevant for hiring decisions are adequately capturedby the system. Finally, a well-constructed call screening summary isgenerated, which includes fitness scores, skills details, and businessneeds signals categorized into highlights, lowlights, and otherconsiderations which empowers the recruiting team to rapidly make hiringdecisions about the candidate.

Embodiments of the omni-channel orchestrated conversation system and thevirtual conversation agent described in this specification differ fromand improve upon currently existing options. In particular, someembodiments differ by providing the ability to ask follow-up questionsregarding past experiences and skills based on the quality of thecandidates' response enables the system to gather detailed informationabout candidates' experiences that can only be possible by today'smechanisms through human-based (i.e., human to human) telephoniccandidate screening. The information derived from the follow-upquestioning helps the omni-channel orchestrated conversation system tobuild (and maintain through automatic and continual machine learningupdates) a robust hard skills fit model. The omni-channel orchestratedconversation system and virtual conversation agent also identify unique“business needs” signals that are beyond a candidate's skills, but whichenables the omni-channel orchestrated conversation system and virtualconversation agent to create a holistic fitness of candidate.Additionally, the omni-channel orchestrated conversation system andvirtual conversation agent generates and provides a well-constructedsummary of the telephonic speech conversation.

The omni-channel orchestrated conversation system and the virtualconversation agent of the present disclosure may be comprised of thefollowing elements. This list of possible constituent elements isintended to be exemplary only and it is not intended that this list beused to limit the omni-channel orchestrated conversation system and thevirtual conversation agent of the present application to just theseelements. Persons having ordinary skill in the art relevant to thepresent disclosure may understand there to be equivalent elements thatmay be substituted within the present disclosure without changing theessential function or operation of the omni-channel orchestratedconversation system and the virtual conversation agent.

1. A virtual conversation agent, which conducts a conversation with ahuman by phone, email, or text. The virtual conversation agent tries toconverge email/text-based conversations with the human into telephonicspeech conversations with the human. The virtual conversation agentfluidly and seamlessly converses with the human throughout theconversation. For example, the virtual conversation agent may bedeployed by a company seeking job candidates and wishing to identifycandidates that are suitable for particular jobs. As such, the virtualconversation agent would conduct a job candidate screening process basedon telephonic speech conversation with one or more human candidates.

2. A conversation transformation module that transforms spokenconversation responses from a candidate into text responses that areanalyzed for quality, etc. That is, the candidate's responses arecaptured and then transformed into categorized structured data. Thecategorized structured data is then fed to the artificial intelligence(AI) engine and machine learning (ML) sub-system for evaluation withrespect to the natural language understanding AI model. The conversationtransformation module is also configured to transform textual follow-upquestions and responses into audible conversational questions andresponses that are audibly spoken by the virtual conversation agent.

3. An artificial intelligence (AI) engine and machine learning (ML)sub-system which build a machine learning model (also referred to as the“natural language understanding AI model”) used to determine quality ofthe candidate's responses and generate possible probing follow-upquestioning and responses. In some embodiments, the AI engine and MLsub-system include a machine learning compositing module that updatesthe machine learning model (or natural language understanding AI model)as a composite of past responses and input data currently fed to themachine learning model (or natural language understanding AI model).

4. A scoring system comprising a fitness scoring and summary generationmodule that scores candidate fitness for jobs or for a particular goalor purpose. Scoring for fitness is based on comparable items (e.g., jobrequirements compared to job experience or education of candidates) andis also based on holistic fitness of the human for a purpose or a goal(e.g., whether, as a candidate, the human candidate would be suitablefor job).

5. A report and communication generation engine (also referred to as a“reporting engine”) that is configured to generate a well-constructedconversation summary with fitness scores, skills details, and businessneeds signals categorized into highlights, lowlights, and otherconsiderations.

6. A previous conversations database that is configured to store data ofprevious conversations between candidates and human or virtual agent.

The omni-channel orchestrated conversation system and the virtualconversation agent of the present disclosure generally works bydeploying the virtual conversation agent to engage in conversation witha human in any of several manners including, without limitation, phoneconversations, email conversations, and text conversations. The virtualconversation also tries to converge email and text conversations intophone conversations. When the conversation is then in phone conversationmode, the virtual conversation agent of some embodiments capturesaudible telephonic responses from the human/candidate. The capturedresponses are transformed by the conversation transformation module ofthe omni-channel orchestrated conversation system from spoken word tounstructured text, and then from unstructured text into structured data.When the conversation transformation module is finished, the structureddata is provided as input to a machine learning model (or “naturallanguage understanding AI model”) built by and utilized for responsequality processing by the artificial intelligence (AI) engine andmachine learning (ML) sub-system. Initially, the machine learning model(or “natural language understanding AI model”) is built by theartificial intelligence (AI) engine and machine learning (ML) sub-systemto calculate fitness values (e.g., candidate fitness values) accordingto several parameters or aspects related to the particular domain offocus (e.g., suitability for a job with particular job requirements) forthe conversation between the virtual conversation agent and the human(candidate). In the domain of employment and candidate recruiting, forexample, the machine learning model (or “natural language understandingAI model”) is built to calculate hard skills fit based on the totalyears of years, skills mentioned by the candidates, years of experienceper skills, duties performed by the candidate in their past job androles performed by the candidate in the past. In parallel, the naturallanguage understanding AI model (or machine learning model) is built bythe artificial intelligence (AI) engine and machine learning (ML)sub-system to calculate the soft skills score based on the textualsentiments of the words spoken by the candidate, the use of filler wordsand the use of impactful words in the telephonic speech conversation.Thereafter, a composite natural language understanding AI model(composite machine learning model) is created that uses the hard skillsfit score, the soft skills score and other business decision points suchas job status, candidate availability for interview, ongoing interviewstatus, alternate offer status, expected salary, negotiable salary,ability to relocate, availability to start on the job, notice period atcurrent company, work authorization, mutual exclusivity to berepresented by a staffing company, to arrive at an overall fit of thecandidate for a role.

To make the omni-channel orchestrated conversation system and thevirtual conversation agent of the present disclosure, a person maydesign, write (code or encode), build, and deploy softwaremodules/programs for the omni-channel orchestrated conversation systemand the virtual conversation agent to operate as a virtual telephonicbot to make telephony calls and then use code criteria to score thecandidate based on qualitative factors. In some embodiments, thesoftware implements the omni-channel orchestrated conversation processfor conducting realtime contextual and fluid conversation with a humanby a virtual conversation agent. In some embodiments, different softwaremodules, software programs, and/or software applications may implementdifferent parts of the omni-channel orchestrated conversation processfor conducting realtime contextual and fluid conversation with a humanby a virtual conversation agent. Details of the omni-channelorchestrated conversation process for conducting realtime contextual andfluid conversation with a human by a virtual conversation agent aredescribed next, by reference to FIGS. 1-3 .

By way of example, FIG. 1 conceptually illustrates a first part of anomni-channel orchestrated conversation process 100 for conductingrealtime contextual and orchestrated omni-channel conversation with ahuman by a virtual conversation agent. As shown in this figure, thefirst part of the omni-channel orchestrated conversation process 100starts with a virtual conversation agent (also referred to as the “bot”)analyzing previous conversation notes on a human candidate (at 110) todecide on whether to initiate a new conversation. For example, the humancandidate may have been in communication previously with another bot ora human recruiter of a company seeking candidates for one or more jobsand, therefore, there may be notes from the previous conversations. Thevirtual conversation agent of some embodiments automatically reviewspast notes before reaching out to candidates, thereby ensuring thatcontextual history with the candidate is maintained for bettercommunication and more fluid, human-like conversation between thevirtual conversation agent and the human candidate. Also, while thisexample for the omni-channel orchestrated conversation process 100pertains to human candidates for jobs, a person of ordinary skill in theart would appreciate that the omni-channel orchestrated conversationprocess for conducting realtime contextual and orchestrated omni-channelconversation with a human by a virtual conversation agent can be adaptedfor any of several needs in pursuit of goals directed toward any ofseveral domains or areas of focus.

Next, the first part of the omni-channel orchestrated conversationprocess 100 proceeds to a step at which the conversation is initiatedbetween the human candidate and the virtual conversation agent (at 120)by way of a phone call, a text message, or an email. Also, theconversation can be initiated by either party—that is, the virtualconversation agent or the human candidate can reach out via one of thechannels of communication (phone, email, text) to initiate aconversation.

In some embodiments, the first part of the omni-channel orchestratedconversation process 100 determines (at 130) whether the human candidateis responsive to the bot. Specifically, if the virtual conversationagent initiated the conversation, is the human candidate making someform of response back to the bot? When the human candidate is responsiveto the bot (‘YES’), then the first part of the omni-channel orchestratedconversation process 100 proceeds forward to a second part of theomni-channel orchestrated conversation process, which is described infurther detail below, by reference to FIG. 2 . On the other hand, whenthe human candidate is determined (at 130) not to be responsive to thevirtual conversation agent (‘NO’), then the first part of theomni-channel orchestrated conversation process 100 proceeds to a stepfor determining (at 140) whether the virtual conversation agentinitiated the conversation by phone or not. When the virtualconversation agent did not initiate the conversation by phone (‘NO’),the first part of the omni-channel orchestrated conversation process 100continues ahead to a step at which the virtual conversation agent sendsa follow-up text message, email message, makes a phone call (or leaves avoicemail message) to the human candidate requesting a response within acertain time frame (at 160). However, when the virtual conversationagent affirmatively did initiate the conversation by phone (‘YES’), thenthe first part of the omni-channel orchestrated conversation process 100proceeds to a step at which the virtual conversation agent leaves avoicemail message if the human candidate does not pick up the phone (at150). Then the omni-channel orchestrated conversation process 100continues forward to the step at which the virtual conversation agentsends a follow-up text message or email message (in this case, no callor voicemail, since a voicemail message was already left from theinitial phone call to the human candidate) requesting a response fromthe human candidate within the particular time frame (at 160).

In some embodiments, the first part of the omni-channel orchestratedconversation process 100 waits for a response from the human candidateand determines (at 170) whether the candidate responded within the timeframe or not. In some embodiments, the first part of the omni-channelorchestrated conversation process 100 automatically listens for responsefrom the candidate by way of a backend communication event triggeringprocess of a server for the omni-channel orchestrated conversationsystem. Thus, when a candidate response is received via email, textmessage, and/or phone call, the first part of the omni-channelorchestrated conversation process 100 of some embodiments checks to seewhether the response is within the time frame and (‘YES’) thentransitions back to the step at which the conversation is initiatedbetween the human candidate and the virtual conversation agent (at 120)and proceeding as noted above. However, when the time frame expires, thefirst part of the omni-channel orchestrated conversation process 100 istriggered by the backend communication event triggering process toproceed to a different step at which the virtual conversation agentgenerates and sends an email summary to the company that deployed thebot (at 180). The email summary indicates that the human candidate wasnot reached and/or did not engage in conversation.

Now, turning back to the determination (at 130) of whether the humancandidate is responsive to the virtual conversation agent or not. Whenthe human candidate is determined (at 130) to be responsive to thevirtual conversation agent (‘YES’), the omni-channel orchestratedconversation process proceeds to steps of a second part of theomni-channel orchestrated conversation process 200, which is nowdescribed by reference to FIG. 2 .

Specifically, and turning to FIG. 2 , a second part of the omni-channelorchestrated conversation process 200 is shown. The second part of theomni-channel orchestrated conversation process 200 continues bydetermining (at 210) whether the conversation is being conducted byphone or not. When the conversation between the human candidate and thevirtual conversation agent is affirmatively being conducted by phone(‘YES’), the second part of the omni-channel orchestrated conversationprocess 200 transitions to steps of a third part of the omni-channelorchestrated conversation process, which is described in detail below,by reference to FIG. 3 .

On the other hand, when the conversation between the human candidate andthe virtual conversation agent is not determined (at 210) to be by phone(‘NO’), then the conversation is being conducted via email or textmessaging. Thus, the second part of the omni-channel orchestratedconversation process 200 performs a step at which the virtualconversation agent tries to converge the email/text conversation into aphone conversation (at 220). However, the virtual conversation agentcannot converge the conversation into a phone conversation without theacquiescence of the human candidate. Therefore, the second part of theomni-channel orchestrated conversation process 200 determines (at 230)whether the human candidate agrees to converge the conversation into aphone conversation by way of a present phone call. When the humancandidate agrees to have a phone call now (‘YES’), the second part ofthe omni-channel orchestrated conversation process 200 carries on to thesteps in the third part of the omni-channel orchestrated conversationprocess described below, by reference to FIG. 3 .

On the other hand, when it is determined (at 230) that the humancandidate does not agree to have a phone call now (‘NO’), then thesecond part of the omni-channel orchestrated conversation process 200determines (at 240) whether the human candidate is requesting a phonecall at a specific later time, instead of a phone call now. When it isdetermined (at 240) that the human candidate is not requesting or askingfor a phone call at a specific later time (‘NO’), then the second partof the omni-channel orchestrated conversation process 200 determines (at250) whether the human candidate is rejecting convergence of theconversation to a phone call conversation. When convergence of theconversation to a phone conversation is not determined (at 250) to berejected by the human candidate (‘NO’), then the second part of theomni-channel orchestrated conversation process 200 transitions back tothe step at which the virtual conversation agent attempts to convergethe conversation into a phone call (at 220), continuing forward asdescribed above. However, when convergence of the conversation to aphone conversation is affirmatively determined (at 250) to be expresslyrejected by the human candidate (‘YES’), then the second part of theomni-channel orchestrated conversation process 200 proceeds to adifferent step at which the virtual conversation agent generates andsends an email summary of the text/email conversation to the companythat deployed the virtual conversation agent when the conversation comesto a closure without a phone call between the virtual conversation agentand the human candidate (at 260).

Turning back to the determination (at 240) of whether the humancandidate is requesting or asking for a phone call at a specific timethat is later than the current time now, and when the human candidatehas indeed requested or asked for a phone call to be scheduled at aspecific, later time (‘YES’), the second part of the omni-channelorchestrated conversation process 200 of some embodiments proceeds to astep at which the virtual conversation agent schedules a phone call (at270) with the human candidate at the specific, later time. Then, afterscheduling the phone call (at 270) with the human candidate at thespecific time, the second part of the omni-channel orchestratedconversation process 200 determines (at 280) whether the specific timeto call the human candidate is the present time (now). The determination(at 280) is checked at intervals of any length. For example, the currenttime may be checked and compared to the specific time every minute orevery second after the virtual conversation agent schedules (at 270) thephone call with the human candidate. Thus, when the specific time tocall the human candidate is not determined (at 280) to be the presenttime (‘NO’), the second part of the omni-channel orchestratedconversation process 200 transitions and repeats in a cycle the timechecking and comparing associated with the determination (at 280) ofwhether the specific time to call the human candidate is the presenttime (now).

On the other hand, when the specific time to call the human candidate isaffirmatively determined (at 280) to be the same as the present time(‘YES’), then second part of the omni-channel orchestrated conversationprocess 200 transitions forward to the steps of the third part of theomni-channel orchestrated conversation process, which is described next,by reference to FIG. 3 .

Specifically referring to FIG. 3 , and continuing where FIG. 2 left off,the third part of the omni-channel orchestrated conversation process 300starts with a phone call that is either currently ongoing (when theoriginal conversation was by phone) or initiated now by the virtualconversation agent to the human candidate (when the originalconversation was by email or text, or the candidate was not responsiveand a message was left followed by a subsequently initiatedconversation). When the conversation is by phone call, live speech ofthe human candidate is captured and the third part of the omni-channelorchestrated conversation process 300 converts the live speech tounstructured text, which is then converted into structured text by theconversation transformation module and fed into the natural languageunderstanding AI model to understand the intent of the spoken (nowtextual) words of the human candidate (at 310). Contemporaneously, thethird part of the omni-channel orchestrated conversation process 300performs a step at which the virtual conversation agent requests consentfrom the human candidate to record the phone call (at 320). Consent bythe candidate is needed in some embodiments to proceed with the phoneconversation. Thus, the third part of the omni-channel orchestratedconversation process 300 determines (at 325) whether the candidateconsents to call recording or not. When the human candidate does notconsent (‘NO’), the third part of the omni-channel orchestratedconversation process 300 transitions back to a step of the second partof the omni-channel orchestrated conversation process 200 in which thevirtual conversation agent generates and sends an email summary of thetext/email conversation to the company that deployed the virtualconversation agent when the conversation comes to a closure without aphone call between the virtual conversation agent and the humancandidate (at 260).

On the other hand, when the human candidate affirmatively consents(‘YES’), the third part of the omni-channel orchestrated conversationprocess 300 begins call recording (at 330) after the consent is receivedfrom the human candidate. As the virtual conversation agent engages inconversation with the human candidate, the third part of theomni-channel orchestrated conversation process 300 proceeds to determineappropriate responses or (next) questions by the virtual conversationagent based on the human candidate's prior response (at 340).Additionally, the third part of the omni-channel orchestratedconversation process 300 analyzes responses by the human candidate andasks contextually appropriate follow-up questions to the human candidate(at 350). In some embodiments, the third part of the omni-channelorchestrated conversation process 300 formulates responses for thevirtual conversation agent (at 360) first at a textual response levelwhich are then converted to audible speech and played back to the humancandidate over the phone. The third part of the omni-channelorchestrated conversation process 300 also determines (at 365) whethernegotiation between the human candidate and the virtual conversationagent is required or needed in connection with the current phoneconversation with the human candidate. When needed, the virtualconversation agent of some embodiments negotiates with the humancandidate (at 370). The ability to negotiate with the human candidate ispossible by the virtual conversation agent whenever the need arisesduring the phone conversation. Also, many of the prior steps of thethird part of the omni-channel orchestrated conversation process 300 maybe performed multiple times—specifically, the steps at which the virtualconversation agent determines appropriate responses or (next) questionsbased on the human candidate's prior response (at 340), the virtualconversation agent analyzes responses by the human candidate and askscontextually appropriate follow-up questions to the human candidate (at350), the virtual conversation agent formulates responses (at 360) at atextual response level which are then converted to audible speech andplayed back to the human candidate over the phone, and the virtualconversation agent determines (at 365) whether negotiation is needed ornot.

In some embodiments, the third part of the omni-channel orchestratedconversation process 300 proceeds with the dialog of the phoneconversation continuing between the human candidate and the virtualconversation agent until the virtual conversation agent determines thatthe phone conversation is over and ends the phone call (at 380). In someembodiments, the virtual conversation agent determines that theconversation is over when the phone call connection to the humancandidate is severed. In some embodiments, the virtual conversationagent determines that the conversation is over based on key phrasesspoken by the human candidate that signal closure. In some embodiments,the virtual conversation agent determines that the conversation is overby asking the human candidate whether there is anything else to add tothe conversation and, upon receiving a response declining the invitationto add more to the conversation, stating a closure phrase. For example,the virtual conversation may ask whether there is “anything more youwant to add or explain about your qualifications or relevance inconnection with the job”, and if the human candidate responds in thenegative, the virtual conversation agent may respond appropriately with“OK, goodbye” or “thank you, goodbye”.

In some embodiments, the third part of the omni-channel orchestratedconversation process 300 then moves on to a step at which the virtualconversation agent automatically sends a follow-up text/email to thehuman candidate to reconnect in the conversation if the phone call was(inadvertently or unexpectedly) closed too soon (at 390). Next, thethird part of the omni-channel orchestrated conversation process 300proceeds to a final step at which the virtual conversation agentgenerates and sends an email summary of the phone conversation to thecompany that deployed the virtual conversation agent (at 395). Then theomni-channel orchestrated conversation process ends.

By way of another example, FIG. 4 conceptually illustrates anomni-channel orchestrated conversation system 400 with a virtualconversation agent for realtime contextual and orchestrated omni-channelconversation with a human in some embodiments. As shown in this figure,the omni-channel orchestrated conversation system 400 includes anomni-channel orchestrated conversation server system 410 and a humanuser 450 who engage in conversation by any of a plurality ofomni-channel communication mechanisms, including an email channel 460, aphone channel 470, and a text messaging channel 480. The omni-channelorchestrated conversation server system 410 includes several componentsand functional elements, namely, a virtual conversation agent 420, aconversation transformation module 430, and an artificial intelligence(AI) engine and machine learning (ML) sub-system 440.

The omni-channel orchestrated conversation system 400 works byconnecting the human user 450 and the virtual conversation agent 420 byway of at least one omni-channel communication mechanism—either theemail channel 460, the phone channel 470, or the text messaging channel480. When the email channel 460 or the text messaging channel 480 isinitiated for the conversation, the virtual conversation agent 420 seeksout a way to converge the conversation with the human user 450 to aphone conversation across the phone channel 470. This may be a simplerequest that is generated by the virtual conversation agent 420 and sentto the human user 450 as an email or in a text message. When the phonechannel 470 is the mechanism of the initiated conversation, or after theemail/text conversation is converged into the phone conversation acrossthe phone channel 470, the virtual conversation agent 420 communicateswith the human user 450 in spoken words that reflect a fluidconversational style, like that of a human operator or human recruiter.While engaging in such audible conversation over the phone channel 470,the virtual conversation agent 420 records or captures the audiblevocalizations of the human user 450 and passes the recorded/capturedaudio of the human user 450 to the conversation transformation module430 which converts the pure audio to unstructured text that is averbatim transcription of the human user 450 audio. The conversationtransformation module 430 of some embodiments then transforms theunstructured text into structured text which is fed into the machinelearning model (natural language understanding AI model) built by theartificial intelligence (AI) engine and machine learning (ML) sub-system440. The natural language understanding AI model (machine learningmodel) is used by the AI engine and ML sub-system 440 to determine thequality of the responses elicited by the human user 450 and generatepossible probing follow-up questioning and responses for the virtualconversation agent 420 to vocalize on the phone channel 470. However,before the virtual conversation agent 420 can audibly express suchprobing follow-up questions or responses to the human user 450 acrossthe phone channel 470, the output from the AI engine and ML sub-system440 (after processing the structured text fed to the natural languageunderstanding AI model) needs to be converted from a textual format toaudio. Thus, the output text from the AI engine and ML sub-system 440 isprocessed by the conversation transformation module 430 to generateaudio of the probing follow-up questions or responses for the virtualconversation agent 420 to play back to the human user 450 across thephone channel 470.

In some embodiments, the artificial intelligence (AI) engine and machinelearning (ML) sub-system 440 further includes a machine learningcompositing module that updates the natural language understanding AImodel as a composite of past responses and input data currently fed tothe natural language understanding AI model.

While not shown in this figure, the omni-channel orchestratedconversation system 400 also includes a previous conversations database,a scoring system, and a report and communication generation engine. Insome embodiments, the previous conversations database is configured tostore data of previous conversations between human users (candidates)and virtual conversation agents or human agents (such as a humanrecruiters at a company, etc.). In some embodiments, the information andnotes associated with the human user 450 which is stored in the previousconversations database is retrieved by the virtual conversation agent420 before initializing the conversation. In this way, the virtualconversation agent 420 has contextual background from which to start theconversation with the human user 450 in a manner that a human wouldexpect from another human.

In some embodiments, the scoring system includes a fitness scoring andsummary generation module that is configured to score the human user'sfitness for a goal or purpose (such as fitness for a particular job)after the AI engine and ML sub-system 440 has evaluated the quality ofthe responses by the human user 450. In some embodiments, scoring forfitness is based on comparable items (e.g., job requirements compared tojob experience or education of the human user 450) and is also based onholistic fitness of the human for a purpose or a goal (e.g., whether, asa candidate, the human user 450 would be suitable for job).

In some embodiments, the report and communication generation engine isconfigured to generate emails, text messages, and well-constructedconversation summaries upon closure of conversations between the virtualconversation agent 420 (any deployed virtual conversation agent) and thehuman user 450 (any human user). In some embodiments, emails, textmessages, summaries include detailed information about the conversation,as well as fitness scores, skills details, and business needs signalscategorized into highlights, lowlights, and other considerations.

Many of the above-described features and applications are implemented assoftware processes that are specified as a set of instructions recordedon a computer readable storage medium (also referred to as computerreadable medium or machine readable medium). When these instructions areexecuted by one or more processing unit(s) (e.g., one or moreprocessors, cores of processors, or other processing units), they causethe processing unit(s) to perform the actions indicated in theinstructions. Examples of computer readable media include, but are notlimited to, CD-ROMs, flash storage (such as USB flash drives or SD flashmemory modules), RAM chips, hard drives (including solid state storagedevices “SSD”), EPROMs, etc. The computer readable media does notinclude carrier waves and electronic signals passing wirelessly or overwired connections.

In this specification, the term “software” is meant to include firmwareresiding in read-only memory or applications stored in magnetic storage,which can be read into memory for processing by a processor. Also,multiple software inventions can be implemented as sub-parts of a largerprogram while remaining distinct software inventions. In someembodiments, multiple software inventions can also be implemented asseparate programs. Finally, any combination of separate programs thattogether implement a software invention described here is within thescope of the invention. In some embodiments, the software programs, wheninstalled to operate on one or more electronic systems, define one ormore specific machine implementations that execute and perform theoperations of the software programs.

FIG. 5 conceptually illustrates an electronic system 500 with which someembodiments of the invention are implemented. The electronic system 500may be a computer, phone (cell phone, mobile phone, smartphone, etc.),PDA (digital assistant device, etc.), tablet computing device, singleboard computer (SBC), server, or any other sort of electronic device orcomputing device. Such an electronic system includes various types ofcomputer readable media and interfaces for various other types ofcomputer readable media. Electronic system 500 includes a bus 505,processing unit(s) 510, a system memory 515, a read-only memory 520, apermanent storage device 525, input devices 530, output devices 535, anda network 540.

The bus 505 collectively represents all system, peripheral, and chipsetbuses that communicatively connect the numerous internal devices of theelectronic system 500. For instance, the bus 505 communicativelyconnects the processing unit(s) 510 with the read-only memory 520, thesystem memory 515, and the permanent storage device 525. From thesevarious memory units, the processing unit(s) 510 retrieves instructionsto execute and data to process in order to execute the processes of theinvention. The processing unit(s) may be a single processor or amulti-core processor in different embodiments.

The read-only-memory (ROM) 520 stores static data and instructions thatare needed by the processing unit(s) 510 and other modules of theelectronic system. The permanent storage device 525, on the other hand,is a read-and-write memory device. This device is a non-volatile memoryunit that stores instructions and data even when the electronic system500 is off. Some embodiments of the invention use a mass-storage device(such as a magnetic or optical disk and its corresponding disk drive) asthe permanent storage device 525.

Other embodiments use a removable storage device (such as a flash drive)as the permanent storage device 525. Like the permanent storage device525, the system memory 515 is a read-and-write memory device. However,unlike storage device 525, the system memory 515 is a volatileread-and-write memory, such as a random access memory. The system memory515 stores some of the instructions and data that the processor needs atruntime. In some embodiments, the invention's processes are stored inthe system memory 515, the permanent storage device 525, and/or theread-only memory 520. For example, the various memory units includeinstructions for processing appearance alterations of animaltemperature. From these various memory units, the processing unit(s) 510retrieves instructions to execute and data to process in order toexecute the processes of some embodiments.

The bus 505 also connects to the input and output devices 530 and 535.The input devices enable the user to communicate information and selectcommands to the electronic system. The input devices 530 includealphanumeric keyboards and pointing devices (also called “cursor controldevices”). The output devices 535 display images generated by theelectronic system 500. The output devices 535 include printers anddisplay devices, such as liquid crystal displays (LCD) or organic lightemitting diode (OLED) displays. Some embodiments include devices such asa touchscreen that functions as both input and output devices.

Finally, as shown in FIG. 5 , bus 505 also couples electronic system 500to a network 540 through a network adapter (not shown). In this manner,the computer can be a part of a network of computers (such as a localarea network (“LAN”), a wide area network (“WAN”), or an intranet), or anetwork of networks (such as the Internet). Any or all components ofelectronic system 500 may be used in conjunction with the invention.

These functions described above can be implemented in digital electroniccircuitry, in computer software, firmware or hardware. The techniquescan be implemented using one or more computer program products.Programmable processors and computers can be packaged or included inmobile devices. The processes may be performed by one or moreprogrammable processors and by one or more set of programmable logiccircuitry. General purpose and special purpose computing and storagedevices can be interconnected through communication networks.

Some embodiments include electronic components, such as microprocessors,storage and memory that store computer program instructions in amachine-readable or computer-readable medium (alternatively referred toas non-transitory computer readable media, computer-readable storagemedia, machine-readable media, machine-readable storage media, or simplyas media). The computer-readable media may store a computer program thatis executable by at least one processing unit and includes sets ofinstructions for performing various operations. Examples of computerprograms or computer code include machine code, such as is produced by acompiler, and files including higher-level code that are executed by acomputer, an electronic component, or a microprocessor using aninterpreter.

I claim:
 1. A non-transitory computer readable medium storing anomni-channel orchestrated conversation program which, when executed byan omni-channel orchestrated conversation server, provides a virtualconversation agent to conduct realtime contextual and orchestratedomni-channel conversation with a human user, said omni-channelorchestrated conversation program comprising sets of instructions for:analyzing previous conversation notes about a human user; initiating anew conversation between the human user and a virtual conversationagent; determining whether the human user is responsive to the virtualconversation agent; requesting a response from the human user by thevirtual conversation agent when the human user is not responsive;determining whether the new conversation between the human user and thevirtual conversation agent is a phone conversation when the human useris responsive to the virtual conversation agent; requesting, by thevirtual conversation agent, to converge the conversation into atelephonic speech conversation by a phone call between the virtualconversation agent and the human user when the new conversation is notthe phone conversation; staring the phone call between the virtualconversation agent and the human user when the human user agrees toconverge the conversation into the telephonic speech conversation;capturing and converting live speech from the telephonic speechconversation to structured text used to understand intent of the humanuser; requesting consent from the human user to record the telephonicspeech conversation; recording the telephonic speech conversation whenthe human user consents to recording; engaging in conversation betweenthe virtual conversation agent and the human user during the telephonicspeech conversation; ending the phone call when the virtual conversationagent determines that the telephone speech conversation is finished; andgenerating and sending an email summary of the telephonic speechconversation to a company that deployed the virtual conversation agent.2. The non-transitory computer readable medium of claim 1, wherein thenew conversation is initiated as one of a phone conversation, an emailconversation, and a text message conversation.
 3. The non-transitorycomputer readable medium of claim 1, wherein the set of instructions forrequesting the response from the human user comprises a set ofinstructions for setting a time duration for the human user to respondto the virtual conversation agent.
 4. The non-transitory computerreadable medium of claim 3, wherein the omni-channel orchestratedconversation program further comprises a set of instructions forgenerating and sending an email summary to the company that deployed thevirtual conversation agent when one of the human user fails to respondwithin the time duration and the human user denies consent to record thetelephonic speech conversation, wherein the email summary indicates thatno conversation between the human user and the virtual conversationagent occurred.
 5. The non-transitory computer readable medium of claim1, wherein the set of instructions for engaging in conversation betweenthe virtual conversation agent and the human user during the telephonicspeech conversation comprises sets of instructions for: determiningappropriate virtual conversation agent responses comprising statementsand questions, wherein the appropriate virtual conversation agentresponses are based on one or more prior responses of the human user;asking, by the virtual conversation agent, contextually appropriatefollow-up questions to the human user; determining whether negotiationbetween the human user and the virtual conversation agent is needed; andnegotiating, by the virtual conversation agent, with the human userwhenever negotiation is needed.
 6. The non-transitory computer readablemedium of claim 1, wherein the omni-channel orchestrated conversationprogram further comprises a set of instructions for automaticallysending a follow-up message to the human user to reconnect the phonecall and re-engage in the telephonic speech conversation when the phonecall was ended too soon.